Pandas Profiling Report

Dataset statistics

Number of variables	36
Number of observations	57694
Missing cells	0
Missing cells (%)	0.0%
Duplicate rows	18308
Duplicate rows (%)	31.7%
Total size in memory	9.7 MiB
Average record size in memory	176.0 B

Variable types

BOOL	17
NUM	16
CAT	3

Reproduction

Analysis started	2021-04-21 05:41:50.022694
Analysis finished	2021-04-21 05:42:44.381787
Duration	54.36 seconds
Version	pandas-profiling v2.8.0
Command line	`pandas_profiling --config_file config.yaml [YOUR_FILE.csv]`
Download configuration	config.yaml

Warnings

`year` has constant value "2019"	Constant
Dataset has 18308 (31.7%) duplicate rows	Duplicates
`bytes_sent` is highly correlated with `bytes` and 2 other fields	High correlation
`bytes` is highly correlated with `bytes_sent` and 3 other fields	High correlation
`packets` is highly correlated with `bytes` and 3 other fields	High correlation
`bytes_received` is highly correlated with `packets` and 1 other fields	High correlation
`ts_ret_a` is highly correlated with `timestamp` and 1 other fields	High correlation
`timestamp` is highly correlated with `ts_ret_a` and 1 other fields	High correlation
`timestamp_relative` is highly correlated with `timestamp` and 1 other fields	High correlation
`packets_sent` is highly correlated with `bytes` and 2 other fields	High correlation
`packets_received` is highly correlated with `bytes_received` and 1 other fields	High correlation
`dayofweek` is highly correlated with `day`	High correlation
`day` is highly correlated with `dayofweek`	High correlation
`avg_bytes_per_packet` is highly correlated with `avg_bytes_received_per_packet`	High correlation
`avg_bytes_received_per_packet` is highly correlated with `avg_bytes_per_packet`	High correlation
`packets_per_rel_time` is highly correlated with `bytes` and 1 other fields	High correlation
`source_port_cat_Dynamic and/or Private Ports` is highly correlated with `source_port_cat_Registered Ports`	High correlation
`source_port_cat_Registered Ports` is highly correlated with `source_port_cat_Dynamic and/or Private Ports`	High correlation
`nat_source_port_cat_Well Known Ports` is highly correlated with `action_allow`	High correlation
`action_allow` is highly correlated with `nat_source_port_cat_Well Known Ports`	High correlation
`nat_dest_port_cat_Registered Ports` is highly correlated with `nat_dest_port_cat_Well Known Ports`	High correlation
`nat_dest_port_cat_Well Known Ports` is highly correlated with `nat_dest_port_cat_Registered Ports`	High correlation
`dayofweek` is highly correlated with `day`	High correlation
`day` is highly correlated with `dayofweek`	High correlation
`bytes` is highly skewed (γ1 = 177.9774935)	Skewed
`bytes_sent` is highly skewed (γ1 = 208.2272139)	Skewed
`bytes_received` is highly skewed (γ1 = 98.96945013)	Skewed
`packets` is highly skewed (γ1 = 138.2226542)	Skewed
`packets_sent` is highly skewed (γ1 = 170.4626862)	Skewed
`packets_received` is highly skewed (γ1 = 112.9164773)	Skewed
`packets_transfer_fraction` is highly skewed (γ1 = 210.2490592)	Skewed
`packets_per_rel_time` is highly skewed (γ1 = 230.2782051)	Skewed
`bytes_received` has 28008 (48.5%) zeros	Zeros
`time_elapsed_sec` has 25027 (43.4%) zeros	Zeros
`packets_received` has 28008 (48.5%) zeros	Zeros
`hour` has 4223 (7.3%) zeros	Zeros
`avg_bytes_received_per_packet` has 28008 (48.5%) zeros	Zeros
`packets_transfer_fraction` has 28008 (48.5%) zeros	Zeros

bytes
Real number (ℝ_≥0)

HIGH CORRELATION
SKEWED

Distinct count	8933
Unique (%)	15.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	98256.55179048081
Minimum	60
Maximum	1269359015
Zeros	0
Zeros (%)	0.0%
Memory size	450.7 KiB

Quantile statistics

Minimum	60
5-th percentile	62
Q1	66
median	168
Q3	734.75
95-th percentile	19962.8
Maximum	1269359015
Range	1269358955
Interquartile range (IQR)	668.75

Descriptive statistics

Standard deviation	5992140.403
Coefficient of variation (CV)	60.98463963
Kurtosis	35813.12688
Mean	98256.55179
Median Absolute Deviation (MAD)	102
Skewness	177.9774935
Sum	5668813499
Variance	3.590574661e+13

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
70	9603	16.6%
66	8402	14.6%
62	4845	8.4%
60	1506	2.6%
146	927	1.6%
177	739	1.3%
168	639	1.1%
199	621	1.1%
184	543	0.9%
183	399	0.7%
Other values (8923)	29470	51.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
60	1506	2.6%
62	4845	8.4%
64	5	< 0.1%
66	8402	14.6%
68	4	< 0.1%

Value	Count	Frequency (%)
1269359015	1	< 0.1%
428935914	2	< 0.1%
127653507	1	< 0.1%
126490514	1	< 0.1%
78198970	2	< 0.1%

bytes_sent
Real number (ℝ_≥0)

HIGH CORRELATION
SKEWED

Distinct count	5794
Unique (%)	10.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	28535.367334558185
Minimum	60
Maximum	948477220
Zeros	0
Zeros (%)	0.0%
Memory size	450.7 KiB

Quantile statistics

Minimum	60
5-th percentile	62
Q1	66
median	88
Q3	192
95-th percentile	4684
Maximum	948477220
Range	948477160
Interquartile range (IQR)	126

Descriptive statistics

Standard deviation	4175410.861
Coefficient of variation (CV)	146.3240621
Kurtosis	46397.45895
Mean	28535.36733
Median Absolute Deviation (MAD)	22
Skewness	208.2272139
Sum	1646319483
Variance	1.743405586e+13

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
70	9922	17.2%
66	8440	14.6%
62	4862	8.4%
94	3345	5.8%
102	2993	5.2%
60	1598	2.8%
86	1297	2.2%
78	1124	1.9%
110	1067	1.8%
146	912	1.6%
Other values (5784)	22134	38.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
60	1598	2.8%
62	4862	8.4%
64	30	0.1%
66	8440	14.6%
68	4	< 0.1%

Value	Count	Frequency (%)
948477220	1	< 0.1%
213443641	2	< 0.1%
122661116	1	< 0.1%
4119315	2	< 0.1%
2726981	1	< 0.1%

bytes_received
Real number (ℝ_≥0)

HIGH CORRELATION
SKEWED
ZEROS

Distinct count	7371
Unique (%)	12.8%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	69721.18445592263
Minimum	0
Maximum	320881795
Zeros	28008
Zeros (%)	48.5%
Memory size	450.7 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	74
Q3	417
95-th percentile	12816.55
Maximum	320881795
Range	320881795
Interquartile range (IQR)	417

Descriptive statistics

Standard deviation	2155345.268
Coefficient of variation (CV)	30.91377871
Kurtosis	12252.47266
Mean	69721.18446
Median Absolute Deviation (MAD)	74
Skewness	98.96945013
Sum	4022494016
Variance	4.645513224e+12

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	28008	48.5%
83	914	1.6%
90	781	1.4%
97	543	0.9%
89	527	0.9%
91	491	0.9%
74	466	0.8%
93	364	0.6%
82	359	0.6%
126	331	0.6%
Other values (7361)	24910	43.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	28008	48.5%
60	319	0.6%
62	78	0.1%
63	12	< 0.1%
64	39	0.1%

Value	Count	Frequency (%)
320881795	1	< 0.1%
215492273	2	< 0.1%
123763533	1	< 0.1%
76157318	2	< 0.1%
58122902	2	< 0.1%

packets
Real number (ℝ_≥0)

HIGH CORRELATION
SKEWED

Distinct count	946
Unique (%)	1.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	110.56787534232329
Minimum	1
Maximum	1036116
Zeros	0
Zeros (%)	0.0%
Memory size	450.7 KiB

Quantile statistics

Minimum	1
5-th percentile	1
Q1	1
median	2
Q3	6
95-th percentile	49
Maximum	1036116
Range	1036115
Interquartile range (IQR)	5

Descriptive statistics

Standard deviation	5900.373747
Coefficient of variation (CV)	53.36426814
Kurtosis	21169.63233
Mean	110.5678753
Median Absolute Deviation (MAD)	1
Skewness	138.2226542
Sum	6379103
Variance	34814410.36

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
1	26476	45.9%
2	14602	25.3%
4	1628	2.8%
6	944	1.6%
10	627	1.1%
19	621	1.1%
20	519	0.9%
18	510	0.9%
16	500	0.9%
17	463	0.8%
Other values (936)	10804	18.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1	26476	45.9%
2	14602	25.3%
3	432	0.7%
4	1628	2.8%
5	128	0.2%

Value	Count	Frequency (%)
1036116	1	< 0.1%
635946	2	< 0.1%
161030	1	< 0.1%
123738	2	< 0.1%
123275	1	< 0.1%

time_elapsed_sec
Real number (ℝ_≥0)

ZEROS

Distinct count	830
Unique (%)	1.4%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	67.10826082434916
Minimum	0
Maximum	10824
Zeros	25027
Zeros (%)	43.4%
Memory size	450.7 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	15
Q3	30
95-th percentile	195
Maximum	10824
Range	10824
Interquartile range (IQR)	30

Descriptive statistics

Standard deviation	309.7017402
Coefficient of variation (CV)	4.614957032
Kurtosis	208.1360332
Mean	67.10826082
Median Absolute Deviation (MAD)	15
Skewness	12.18161335
Sum	3871744
Variance	95915.16787

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	25027	43.4%
30	9132	15.8%
31	2886	5.0%
29	1871	3.2%
15	1595	2.8%
5	1205	2.1%
16	862	1.5%
8	758	1.3%
1200	529	0.9%
32	462	0.8%
Other values (820)	13367	23.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	25027	43.4%
1	33	0.1%
2	5	< 0.1%
3	1	< 0.1%
4	309	0.5%

Value	Count	Frequency (%)
10824	1	< 0.1%
9283	1	< 0.1%
9115	1	< 0.1%
8912	1	< 0.1%
8169	1	< 0.1%

timestamp
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct count	32806
Unique (%)	56.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	1552632004606.116
Minimum	1552573800000
Maximum	1552695187200
Zeros	0
Zeros (%)	0.0%
Memory size	450.7 KiB

Quantile statistics

Minimum	1.5525738e+12
5-th percentile	1.552578755e+12
Q1	1.552601348e+12
median	1.552629462e+12
Q3	1.552663132e+12
95-th percentile	1.552688834e+12
Maximum	1.552695187e+12
Range	121387200
Interquartile range (IQR)	61784064

Descriptive statistics

Standard deviation	35980075.78
Coefficient of variation (CV)	2.317360177e-05
Kurtosis	-1.257592663
Mean	1.552632005e+12
Median Absolute Deviation (MAD)	31407336
Skewness	0.09821640971
Sum	8.957755087e+16
Variance	1.294565853e+15

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
1.552583288e+12	406	0.7%
1.552583583e+12	328	0.6%
1.552580488e+12	161	0.3%
1.552606678e+12	151	0.3%
1.552577633e+12	149	0.3%
1.552600289e+12	146	0.3%
1.552583075e+12	145	0.3%
1.552639718e+12	144	0.2%
1.552578688e+12	143	0.2%
1.55260559e+12	142	0.2%
Other values (32796)	55779	96.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1.5525738e+12	1	< 0.1%
1.552573801e+12	1	< 0.1%
1.55257383e+12	1	< 0.1%
1.552573831e+12	1	< 0.1%
1.552573831e+12	1	< 0.1%

Value	Count	Frequency (%)
1.552695187e+12	1	< 0.1%
1.552695186e+12	1	< 0.1%
1.552695185e+12	1	< 0.1%
1.552695185e+12	1	< 0.1%
1.552695181e+12	21	< 0.1%

ts_ret_a
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct count	29239
Unique (%)	50.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	54370635.3100149
Minimum	0
Maximum	103540944
Zeros	1
Zeros (%)	< 0.1%
Memory size	450.7 KiB

Quantile statistics

Minimum	0
5-th percentile	4955148
Q1	27548232
median	55662120
Q3	83831166
95-th percentile	95950008
Maximum	103540944
Range	103540944
Interquartile range (IQR)	56282934

Descriptive statistics

Standard deviation	31030807.56
Coefficient of variation (CV)	0.5707273307
Kurtosis	-1.3557186
Mean	54370635.31
Median Absolute Deviation (MAD)	28113888
Skewness	-0.1629404262
Sum	3.136859434e+12
Variance	9.629110177e+14

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
9488040	406	0.7%
9782736	328	0.6%
88852968	279	0.5%
92161056	270	0.5%
83033328	264	0.5%
80732496	259	0.4%
95452656	249	0.4%
80081664	248	0.4%
83588904	245	0.4%
84663504	237	0.4%
Other values (29229)	54909	95.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	1	< 0.1%
1128	1	< 0.1%
30312	1	< 0.1%
30696	1	< 0.1%
31080	1	< 0.1%

Value	Count	Frequency (%)
103540944	3	< 0.1%
103539096	1	< 0.1%
103538736	1	< 0.1%
103538544	1	< 0.1%
103537824	9	< 0.1%

timestamp_relative
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct count	29239
Unique (%)	50.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	2265443.1379172876
Minimum	0
Maximum	4314206
Zeros	1
Zeros (%)	< 0.1%
Memory size	450.7 KiB

Quantile statistics

Minimum	0
5-th percentile	206464.5
Q1	1147843
median	2319255
Q3	3492965.25
95-th percentile	3997917
Maximum	4314206
Range	4314206
Interquartile range (IQR)	2345122.25

Descriptive statistics

Standard deviation	1292950.315
Coefficient of variation (CV)	0.5707273307
Kurtosis	-1.3557186
Mean	2265443.138
Median Absolute Deviation (MAD)	1171412
Skewness	-0.1629404262
Sum	1.307024764e+11
Variance	1.671720517e+12

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
395335	406	0.7%
407614	328	0.6%
3702207	279	0.5%
3840044	270	0.5%
3459722	264	0.5%
3363854	259	0.4%
3977194	249	0.4%
3336736	248	0.4%
3482871	245	0.4%
3527646	237	0.4%
Other values (29229)	54909	95.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	1	< 0.1%
47	1	< 0.1%
1263	1	< 0.1%
1279	1	< 0.1%
1295	1	< 0.1%

Value	Count	Frequency (%)
4314206	3	< 0.1%
4314129	1	< 0.1%
4314114	1	< 0.1%
4314106	1	< 0.1%
4314076	9	< 0.1%

packets_sent
Real number (ℝ_≥0)

HIGH CORRELATION
SKEWED

Distinct count	631
Unique (%)	1.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	47.83651679550733
Minimum	1
Maximum	747520
Zeros	0
Zeros (%)	0.0%
Memory size	450.7 KiB

Quantile statistics

Minimum	1
5-th percentile	1
Q1	1
median	1
Q3	3
95-th percentile	24
Maximum	747520
Range	747519
Interquartile range (IQR)	2

Descriptive statistics

Standard deviation	3647.842009
Coefficient of variation (CV)	76.25643031
Kurtosis	32341.03893
Mean	47.8365168
Median Absolute Deviation (MAD)	0
Skewness	170.4626862
Sum	2759880
Variance	13306751.33

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
1	40049	69.4%
2	2464	4.3%
3	1322	2.3%
11	1082	1.9%
6	1040	1.8%
8	869	1.5%
5	868	1.5%
10	848	1.5%
9	845	1.5%
7	839	1.5%
Other values (621)	7468	12.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1	40049	69.4%
2	2464	4.3%
3	1322	2.3%
4	681	1.2%
5	868	1.5%

Value	Count	Frequency (%)
747520	1	< 0.1%
308738	2	< 0.1%
82907	1	< 0.1%
43917	2	< 0.1%
41466	1	< 0.1%

packets_received
Real number (ℝ_≥0)

HIGH CORRELATION
SKEWED
ZEROS

Distinct count	792
Unique (%)	1.4%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	62.28677158803342
Minimum	0
Maximum	327208
Zeros	28008
Zeros (%)	48.5%
Memory size	450.7 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	1
Q3	2
95-th percentile	25
Maximum	327208
Range	327208
Interquartile range (IQR)	2

Descriptive statistics

Standard deviation	2457.860601
Coefficient of variation (CV)	39.46039485
Kurtosis	14258.49584
Mean	62.28677159
Median Absolute Deviation (MAD)	1
Skewness	112.9164773
Sum	3593573
Variance	6041078.732

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	28008	48.5%
1	14240	24.7%
2	1603	2.8%
8	1117	1.9%
3	1038	1.8%
9	1020	1.8%
7	886	1.5%
10	861	1.5%
5	823	1.4%
4	742	1.3%
Other values (782)	7356	12.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	28008	48.5%
1	14240	24.7%
2	1603	2.8%
3	1038	1.8%
4	742	1.3%

Value	Count	Frequency (%)
327208	2	< 0.1%
288596	1	< 0.1%
81809	1	< 0.1%
79821	2	< 0.1%
78123	1	< 0.1%

day
Categorical

HIGH CORRELATION
HIGH CORRELATION

Distinct count	3
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	450.7 KiB

15	39787
14	14140
16	3767

Common Values
Length

Value	Count	Frequency (%)
15	39787	69.0%
14	14140	24.5%
16	3767	6.5%

Length

Max length	2
Median length	2
Mean length	2
Min length	2

hour
Real number (ℝ_≥0)

ZEROS

Distinct count	24
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	12.48472978125975
Minimum	0
Maximum	23
Zeros	4223
Zeros (%)	7.3%
Memory size	450.7 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	5
median	15
Q3	19
95-th percentile	23
Maximum	23
Range	23
Interquartile range (IQR)	14

Descriptive statistics

Standard deviation	7.718892527
Coefficient of variation (CV)	0.6182666875
Kurtosis	-1.362438247
Mean	12.48472978
Median Absolute Deviation (MAD)	6
Skewness	-0.304946079
Sum	720294
Variance	59.58130184

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	4223	7.3%
17	4030	7.0%
19	4006	6.9%
1	3595	6.2%
21	3540	6.1%
23	3449	6.0%
18	3301	5.7%
22	3229	5.6%
20	2924	5.1%
16	2834	4.9%
Other values (14)	22563	39.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	4223	7.3%
1	3595	6.2%
2	2173	3.8%
3	1794	3.1%
4	1868	3.2%

Value	Count	Frequency (%)
23	3449	6.0%
22	3229	5.6%
21	3540	6.1%
20	2924	5.1%
19	4006	6.9%

year
Categorical

CONSTANT
REJECTED

Distinct count	1
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	450.7 KiB

2019	57694

Common Values
Length

Value	Count	Frequency (%)
2019	57694	100.0%

Length

Max length	4
Median length	4
Mean length	4
Min length	4

dayofweek
Categorical

HIGH CORRELATION
HIGH CORRELATION

Distinct count	3
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	450.7 KiB

4	39787
3	14140
5	3767

Common Values
Length

Value	Count	Frequency (%)
4	39787	69.0%
3	14140	24.5%
5	3767	6.5%

Length

Max length	1
Median length	1
Mean length	1
Min length	1

avg_bytes_received_per_packet
Real number (ℝ_≥0)

HIGH CORRELATION
ZEROS

Distinct count	8403
Unique (%)	14.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	167.28404356197723
Minimum	0.0
Maximum	1513.9154804270463
Zeros	28008
Zeros (%)	48.5%
Memory size	450.7 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	61.5
Q3	146
95-th percentile	769
Maximum	1513.91548
Range	1513.91548
Interquartile range (IQR)	146

Descriptive statistics

Standard deviation	297.481276
Coefficient of variation (CV)	1.778300367
Kurtosis	6.169713093
Mean	167.2840436
Median Absolute Deviation (MAD)	61.5
Skewness	2.489507765
Sum	9651285.609
Variance	88495.10957

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	28008	48.5%
83	942	1.6%
90	857	1.5%
60	640	1.1%
97	580	1.0%
89	547	0.9%
91	530	0.9%
74	521	0.9%
82	385	0.7%
93	377	0.7%
Other values (8393)	24307	42.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	28008	48.5%
60	640	1.1%
60.46153846	2	< 0.1%
60.5	1	< 0.1%
60.54545455	1	< 0.1%

Value	Count	Frequency (%)
1513.91548	1	< 0.1%
1513.472247	1	< 0.1%
1513.450471	1	< 0.1%
1513.389054	1	< 0.1%
1513.135567	1	< 0.1%

avg_bytes_sent_per_packet
Real number (ℝ_≥0)

Distinct count	8033
Unique (%)	13.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	103.82129485497478
Minimum	30.0
Maximum	1514.0
Zeros	0
Zeros (%)	0.0%
Memory size	450.7 KiB

Quantile statistics

Minimum	30
5-th percentile	50.5
Q1	66
median	74
Q3	102.8869904
95-th percentile	234
Maximum	1514
Range	1484
Interquartile range (IQR)	36.88699041

Descriptive statistics

Standard deviation	88.56841522
Coefficient of variation (CV)	0.8530852495
Kurtosis	55.62895297
Mean	103.8212949
Median Absolute Deviation (MAD)	14
Skewness	5.998635781
Sum	5989865.785
Variance	7844.364175

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
70	8991	15.6%
66	8321	14.4%
62	4878	8.5%
94	3300	5.7%
102	2960	5.1%
60	1532	2.7%
86	1290	2.2%
78	1140	2.0%
110	1042	1.8%
35	945	1.6%
Other values (8023)	23295	40.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
30	78	0.1%
32	15	< 0.1%
33	139	0.2%
35	945	1.6%
35.5	1	< 0.1%

Value	Count	Frequency (%)
1514	4	< 0.1%
1484	3	< 0.1%
1479.502527	1	< 0.1%
1468	1	< 0.1%
1446	1	< 0.1%

avg_bytes_per_packet
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct count	10337
Unique (%)	17.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	158.64381284697097
Minimum	8.393298245614035
Maximum	1514.0
Zeros	0
Zeros (%)	0.0%
Memory size	450.7 KiB

Quantile statistics

Minimum	8.393298246
5-th percentile	60
Q1	66
median	83
Q3	146
95-th percentile	505.9038462
Maximum	1514
Range	1505.606702
Interquartile range (IQR)	80

Descriptive statistics

Standard deviation	181.8829688
Coefficient of variation (CV)	1.146486368
Kurtosis	9.671593626
Mean	158.6438128
Median Absolute Deviation (MAD)	21
Skewness	2.920567208
Sum	9152796.138
Variance	33081.41433

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
70	8794	15.2%
66	8290	14.4%
62	4874	8.4%
60	1511	2.6%
146	924	1.6%
35	831	1.4%
88.5	741	1.3%
84	646	1.1%
99.5	614	1.1%
92	540	0.9%
Other values (10327)	29929	51.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
8.393298246	1	< 0.1%
30	2	< 0.1%
33	131	0.2%
35	831	1.4%
38.5	1	< 0.1%

Value	Count	Frequency (%)
1514	4	< 0.1%
1484	3	< 0.1%
1468	1	< 0.1%
1347.447044	1	< 0.1%
1345	1	< 0.1%

packets_transfer_fraction
Real number (ℝ_≥0)

SKEWED
ZEROS

Distinct count	1920
Unique (%)	3.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	0.5847293159254474
Minimum	0.0
Maximum	863.6666666666666
Zeros	28008
Zeros (%)	48.5%
Memory size	450.7 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0.5172413793
Q3	1
95-th percentile	1.4
Maximum	863.6666667
Range	863.6666667
Interquartile range (IQR)	1

Descriptive statistics

Standard deviation	3.767052497
Coefficient of variation (CV)	6.44238692
Kurtosis	47817.44462
Mean	0.5847293159
Median Absolute Deviation (MAD)	0.5172413793
Skewness	210.2490592
Sum	33735.37315
Variance	14.19068451

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	28008	48.5%
1	16914	29.3%
1.5	599	1.0%
2	593	1.0%
1.25	504	0.9%
1.333333333	458	0.8%
3	450	0.8%
1.2	435	0.8%
1.375	331	0.6%
1.222222222	306	0.5%
Other values (1910)	9096	15.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	28008	48.5%
0.005707762557	1	< 0.1%
0.008415147265	1	< 0.1%
0.01404494382	1	< 0.1%
0.03174603175	1	< 0.1%

Value	Count	Frequency (%)
863.6666667	1	< 0.1%
139.8333333	1	< 0.1%
100.7857143	2	< 0.1%
72.64102564	1	< 0.1%
58.125	2	< 0.1%

packets_per_rel_time
Real number (ℝ_≥0)

HIGH CORRELATION
SKEWED

Distinct count	29459
Unique (%)	51.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	5.7025604860322e-05
Minimum	0.0
Maximum	1.3789083396358672
Zeros	1
Zeros (%)	< 0.1%
Memory size	450.7 KiB

Quantile statistics

Minimum	0
5-th percentile	2.547790174e-07
Q1	3.161913612e-07
median	7.548559887e-07
Q3	3.051573293e-06
95-th percentile	2.672168026e-05
Maximum	1.37890834
Range	1.37890834
Interquartile range (IQR)	2.735381931e-06

Descriptive statistics

Standard deviation	0.005825808768
Coefficient of variation (CV)	102.1612797
Kurtosis	54402.44306
Mean	5.702560486e-05
Median Absolute Deviation (MAD)	4.860390012e-07
Skewness	230.2782051
Sum	3.290035247
Variance	3.39400478e-05

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
2.529500297e-06	406	0.7%
2.453301408e-06	327	0.6%
2.604136828e-07	270	0.5%
2.972780626e-07	258	0.4%
2.890405645e-07	256	0.4%
2.514335484e-07	249	0.4%
2.996940723e-07	248	0.4%
2.871194483e-07	243	0.4%
2.701091538e-07	239	0.4%
2.912111881e-07	233	0.4%
Other values (29449)	54965	95.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	1	< 0.1%
2.317923623e-07	3	< 0.1%
2.317993471e-07	9	< 0.1%
2.318010127e-07	1	< 0.1%
2.318099325e-07	1	< 0.1%

Value	Count	Frequency (%)
1.37890834	1	< 0.1%
0.1478402313	1	< 0.1%
0.08544265472	1	< 0.1%
0.07759012176	2	< 0.1%
0.06668456856	1	< 0.1%

action_allow
Boolean

HIGH CORRELATION

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	56.3 KiB

1	32969
0	24725

Frequency Table

Value	Count	Frequency (%)
1	32969	57.1%
0	24725	42.9%

action_deny
Boolean

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	56.3 KiB

0	44634
1	13060

Frequency Table

Value	Count	Frequency (%)
0	44634	77.4%
1	13060	22.6%

action_drop
Boolean

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	56.3 KiB

0	46074
1	11620

Frequency Table

Value	Count	Frequency (%)
0	46074	79.9%
1	11620	20.1%

action_reset-both
Boolean

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	56.3 KiB

0	57649
1	45

Frequency Table

Value	Count	Frequency (%)
0	57649	99.9%
1	45	0.1%

is_dest_port_53
Boolean

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	450.7 KiB

0	44281
1	13413

Frequency Table

Value	Count	Frequency (%)
0	44281	76.8%
1	13413	23.2%

source_port_cat_Well Known Ports
Boolean

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	56.3 KiB

0	57261
1	433

Frequency Table

Value	Count	Frequency (%)
0	57261	99.2%
1	433	0.8%

source_port_cat_Registered Ports
Boolean

HIGH CORRELATION

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	56.3 KiB

0	43869
1	13825

Frequency Table

Value	Count	Frequency (%)
0	43869	76.0%
1	13825	24.0%

source_port_cat_Dynamic and/or Private Ports
Boolean

HIGH CORRELATION

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	56.3 KiB

1	43436
0	14258

Frequency Table

Value	Count	Frequency (%)
1	43436	75.3%
0	14258	24.7%

dest_port_cat_Well Known Ports
Boolean

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	56.3 KiB

1	39814
0	17880

Frequency Table

Value	Count	Frequency (%)
1	39814	69.0%
0	17880	31.0%

dest_port_cat_Registered Ports
Boolean

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	56.3 KiB

0	44790
1	12904

Frequency Table

Value	Count	Frequency (%)
0	44790	77.6%
1	12904	22.4%

dest_port_cat_Dynamic and/or Private Ports
Boolean

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	56.3 KiB

0	52718
1	4976

Frequency Table

Value	Count	Frequency (%)
0	52718	91.4%
1	4976	8.6%

nat_source_port_cat_Well Known Ports
Boolean

HIGH CORRELATION

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	56.3 KiB

0	32520
1	25174

Frequency Table

Value	Count	Frequency (%)
0	32520	56.4%
1	25174	43.6%

nat_source_port_cat_Registered Ports
Boolean

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	56.3 KiB

0	34056
1	23638

Frequency Table

Value	Count	Frequency (%)
0	34056	59.0%
1	23638	41.0%

nat_source_port_cat_Dynamic and/or Private Ports
Boolean

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	56.3 KiB

0	48812
1	8882

Frequency Table

Value	Count	Frequency (%)
0	48812	84.6%
1	8882	15.4%

nat_dest_port_cat_Well Known Ports
Boolean

HIGH CORRELATION

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	56.3 KiB

1	52302
0	5392

Frequency Table

Value	Count	Frequency (%)
1	52302	90.7%
0	5392	9.3%

nat_dest_port_cat_Registered Ports
Boolean

HIGH CORRELATION

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	56.3 KiB

0	53183
1	4511

Frequency Table

Value	Count	Frequency (%)
0	53183	92.2%
1	4511	7.8%

nat_dest_port_cat_Dynamic and/or Private Ports
Boolean

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	56.3 KiB

0	56813
1	881

Frequency Table

Value	Count	Frequency (%)
0	56813	98.5%
1	881	1.5%

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

Cramér's V (φc)

Cramér's V is an association measure for nominal random variables. The coefficient ranges from 0 to 1, with 0 indicating independence and 1 indicating perfect association. The empirical estimators used for Cramér's V have been proved to be biased, even for large samples. We use a bias-corrected measure that has been proposed by Bergsma in 2013 that can be found here.

Count
Matrix

First rows

	bytes	bytes_sent	bytes_received	packets	time_elapsed_sec	timestamp	ts_ret_a	timestamp_relative	packets_sent	packets_received	day	hour	year	dayofweek	avg_bytes_received_per_packet	avg_bytes_sent_per_packet	avg_bytes_per_packet	packets_transfer_fraction	packets_per_rel_time	action_allow	action_deny	action_drop	is_dest_port_53	source_port_cat_Registered Ports	source_port_cat_Dynamic and/or Private Ports	dest_port_cat_Well Known Ports	dest_port_cat_Registered Ports	dest_port_cat_Dynamic and/or Private Ports	nat_source_port_cat_Well Known Ports	nat_source_port_cat_Registered Ports	nat_dest_port_cat_Well Known Ports
0	284	96	188	4	31	1552591724448	17924448	746852	2	2	14	21	2019	3	94.0000	48.000000	71.000000	1.00	2.677907e-06	1	0	0	1	0	1	1	0	0	0	1	1
1	70	70	0	1	0	1552591785360	17985360	749390	1	0	14	21	2019	3	0.0000	70.000000	70.000000	0.00	1.334419e-06	0	0	1	0	0	1	1	0	0	1	0	1
2	17376	1687	15689	28	40	1552675567368	101767368	4240307	12	16	15	20	2019	4	980.5625	140.583333	620.571429	0.75	2.829984e-06	1	0	0	0	0	1	1	0	0	0	1	1
3	66	66	0	1	0	1552655063160	81263160	3385965	1	0	15	15	2019	4	0.0000	66.000000	66.000000	0.00	2.953368e-07	0	0	1	0	0	1	1	0	0	1	0	1
4	579	88	491	4	35	1552658929680	85129680	3547070	2	2	15	16	2019	4	245.5000	44.000000	144.750000	1.00	5.638457e-07	1	0	0	1	0	1	1	0	0	0	1	1
5	60	60	0	1	0	1552591203120	17403120	725130	1	0	14	21	2019	3	0.0000	60.000000	60.000000	0.00	1.379063e-06	0	1	0	0	1	0	0	1	0	1	0	1
6	66	66	0	1	0	1552614416184	40616184	1692341	1	0	15	3	2019	4	0.0000	66.000000	66.000000	0.00	5.908975e-07	0	1	0	0	0	1	0	0	1	1	0	1
7	232	77	155	2	2	1552634107848	60307848	2512827	1	1	15	9	2019	4	155.0000	77.000000	116.000000	1.00	3.979582e-07	1	0	0	1	0	1	1	0	0	1	0	1
8	62	62	0	1	0	1552637852736	64052736	2668864	1	0	15	10	2019	4	0.0000	62.000000	62.000000	0.00	3.746913e-07	0	1	0	0	1	0	0	1	0	1	0	1
9	146	146	0	1	0	1552689052248	89465688	3727737	1	0	16	0	2019	5	0.0000	146.000000	146.000000	0.00	2.682593e-07	0	1	0	0	1	0	0	0	1	1	0	1

Last rows

	bytes	bytes_sent	bytes_received	packets	time_elapsed_sec	timestamp	ts_ret_a	timestamp_relative	packets_sent	packets_received	day	hour	year	dayofweek	avg_bytes_received_per_packet	avg_bytes_sent_per_packet	avg_bytes_per_packet	packets_transfer_fraction	packets_per_rel_time	action_allow	action_deny	action_drop	is_dest_port_53	source_port_cat_Registered Ports	source_port_cat_Dynamic and/or Private Ports	dest_port_cat_Well Known Ports	dest_port_cat_Registered Ports	dest_port_cat_Dynamic and/or Private Ports	nat_source_port_cat_Well Known Ports	nat_source_port_cat_Registered Ports	nat_source_port_cat_Dynamic and/or Private Ports	nat_dest_port_cat_Well Known Ports
57684	66	66	0	1	0	1552631371392	57571392	2398808	1	0	15	8	2019	4	0.000000	66.000000	66.000000	0.000000	4.168737e-07	0	1	0	0	1	0	0	0	1	1	0	0	1
57685	66	66	0	1	0	1552649392824	75592824	3149701	1	0	15	13	2019	4	0.000000	66.000000	66.000000	0.000000	3.174905e-07	0	0	1	0	0	1	1	0	0	1	0	0	1
57686	299	102	197	2	30	1552659707664	85907664	3579486	1	1	15	16	2019	4	197.000000	102.000000	149.500000	1.000000	2.793697e-07	1	0	0	1	0	1	1	0	0	0	1	0	1
57687	7083	2786	4297	24	81	1552636579248	62779248	2615802	14	10	15	9	2019	4	429.700000	199.000000	295.125000	1.400000	5.352087e-06	1	0	0	0	0	1	1	0	0	0	0	1	1
57688	60	60	0	1	0	1552603711800	29911800	1246325	1	0	15	0	2019	4	0.000000	60.000000	60.000000	0.000000	8.023589e-07	0	1	0	0	1	0	1	0	0	1	0	0	1
57689	66	66	0	1	0	1552644393480	70593480	2941395	1	0	15	12	2019	4	0.000000	66.000000	66.000000	0.000000	3.399747e-07	0	1	0	0	0	1	0	1	0	1	0	0	1
57690	130	66	64	2	4	1552637540088	63740088	2655837	1	1	15	10	2019	4	64.000000	66.000000	65.000000	1.000000	3.765291e-07	1	0	0	0	1	0	1	0	0	0	1	0	1
57691	66	66	0	1	0	1552639165104	65365104	2723546	1	0	15	10	2019	4	0.000000	66.000000	66.000000	0.000000	3.671684e-07	0	1	0	0	0	1	0	1	0	1	0	0	1
57692	70	70	0	1	0	1552682000976	82414416	3433934	1	0	15	22	2019	4	0.000000	70.000000	70.000000	0.000000	2.912112e-07	0	0	1	0	0	1	1	0	0	1	0	0	1
57693	239209	233954	5255	28500	88	1552695187200	95600640	3983360	2822	28	16	2	2019	5	187.678571	82.903614	8.393298	100.785714	7.084471e-04	1	0	0	1	1	0	1	0	0	0	0	1	1

Most frequent

	bytes	bytes_sent	packets	timestamp	ts_ret_a	timestamp_relative	packets_sent	day	hour	year	dayofweek	avg_bytes_sent_per_packet	avg_bytes_per_packet	packets_per_rel_time	action_deny	action_drop	source_port_cat_Dynamic and/or Private Ports	dest_port_cat_Well Known Ports	dest_port_cat_Registered Ports	nat_source_port_cat_Well Known Ports	nat_dest_port_cat_Well Known Ports	count
393	62	62	1	1552583288040	9488040	395335	1	14	19	2019	3	62.0	62.0	2.529500e-06	1	0	1	0	1	1	1	384
396	62	62	1	1552583582736	9782736	407614	1	14	19	2019	3	62.0	62.0	2.453301e-06	1	0	1	0	1	1	1	226
2530	70	70	1	1552620969672	47169672	1965403	1	15	5	2019	4	70.0	70.0	5.088015e-07	0	1	1	1	0	1	1	49
2742	70	70	1	1552679668224	80081664	3336736	1	15	21	2019	4	70.0	70.0	2.996941e-07	0	1	1	1	0	1	1	48
2505	70	70	1	1552614416184	40616184	1692341	1	15	3	2019	4	70.0	70.0	5.908975e-07	0	1	1	1	0	1	1	47
2752	70	70	1	1552682619888	83033328	3459722	1	15	22	2019	4	70.0	70.0	2.890406e-07	0	1	1	1	0	1	1	47
2625	70	70	1	1552648629816	74829816	3117909	1	15	13	2019	4	70.0	70.0	3.207278e-07	0	1	1	1	0	1	1	46
2660	70	70	1	1552656833328	83033328	3459722	1	15	15	2019	4	70.0	70.0	2.890406e-07	0	1	1	1	0	1	1	46
2507	70	70	1	1552614842664	41042664	1710111	1	15	3	2019	4	70.0	70.0	5.847574e-07	0	1	1	1	0	1	1	45
2743	70	70	1	1552680319056	80732496	3363854	1	15	22	2019	4	70.0	70.0	2.972781e-07	0	1	1	1	0	1	1	45

Overview

Variables

Interactions

Correlations

Pearson's r

Spearman's ρ

Kendall's τ

Phik (φk)

Cramér's V (φc)

Missing values

Sample

First rows

Last rows

Duplicate rows

Most frequent